Imaging apparatus and method for controlling same

ABSTRACT

The imaging element of the imaging apparatus is provided with pixels respectively having multiple photoelectric conversion units that generate image signals by photoelectrically converting light fluxes that transit different regions dividing an exit pupil of an imaging optical system to a single micro lens. The imaging apparatus generates left-eye image data and right-eye image data for three-dimensional display based on image signals output by the imaging element, and generates combined image data for two-dimensional display by additively combining the left-eye image data and the right-eye image data. The imaging apparatus compresses the combined image data at a first compression rate, compresses the left-eye image data and the right-eye image data at a second compression rate that is higher than the first compression rate, and records the respectively compressed image data in the same image file.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging apparatus, and an imagingapparatus control method.

2. Description of the Related Art

In recent years, there has been a rapid increase in the prevalence ofdevices associated with stereoscopic images such as three-dimensional(3D) cinema, 3D display, and the like. Conventionally, photographing ofstereoscopic images has been carried out by film cameras and the like.However, with the prevalence of digital imaging apparatuses, it hasbecome common to photograph original images for generation ofstereoscopic images using digital cameras, digital video cameras, andthe like.

As a mechanism by which a user views a stereoscopic image, data for aright-eye image and a left-eye image with parallax in the left-rightdirection is prepared so as to correspond to an image of the objectviewed with the left eye and an image of the object viewed with theright eye. A user can view stereoscopic images by viewing each imagewith his/her right eye and left eye, respectively. Examples of such amethod include methods for dividing a parallax image to be viewed, suchas a parallax barrier method, a lenticular method, or the like. There isalso known to be a method for providing different images to the left eyeand the right eye of a user via a filter having differentcharacteristics between the left and right sides thereof.

On the other hand, as a method for capturing an image which is viewableas a stereoscopic image, Japanese Patent Laid-Open No. 58-24105discloses a method for simultaneously capturing images at differentviewpoints. Japanese Patent Laid-Open No. 58-24105 discloses asolid-state imaging element in which multiple micro lenses are formed,and at least one pair of photodiodes is arranged close to each of themicro lenses.

Of the pair of photodiodes, a first image signal is obtained from theoutput of one photodiode and a second image signal is obtained from theoutput of the other photodiode. A user can view a stereoscopic image byusing the first and second image signals as a left-eye image and aright-eye image, respectively. It would also be conceivable to have animaging apparatus in which a combined image is generated based on theaforementioned first image signal and second image signal, and thegenerated combined image is recorded as an image for two-dimensionaldisplay.

In this instance, although an imaging apparatus could manage a left-eyeimage and a right-eye image as separate files, it would becomeimpossible to view a stereoscopic image if one of the files were lost.Japanese Patent Laid-Open No. 2010-200354 discloses an image filegeneration method which records a left-eye image and a right-eye imagein the same file.

Applying the image file generation method disclosed in Japanese PatentLaid-Open No. 2010-200354, it would be conceivable to have an imagingapparatus in which a left-eye image, a right-eye image, and a combinedimage are recorded in one file. However, the file size would be large ifsuch an imaging apparatus were used.

SUMMARY OF THE INVENTION

The imaging apparatus of the present invention is an imaging apparatusin which a left-eye image, a right-eye image, and a combined image dataare recorded in one file, and the file size of the recorded image datais minimized.

The imaging apparatus of an aspect of the present invention includes: animaging element provided with pixels which respectively have multiplephotoelectric conversion units that generate image signals byphotoelectrically converting light fluxes that transit different regionsof exit pupil of an imaging optical system to one micro lens; an imagegeneration unit configured to generate left-eye image data and right-eyeimage data for three-dimensional display based on image signals outputby the imaging element, and generate combined image data fortwo-dimensional display by additively combining the generated left-eyeimage data and right-eye image data; an image compression unitconfigured to compress the combined image data at a predetermined firstcompression rate, and compress the left-eye image data and the right-eyeimage data at a second compression rate that is higher than the firstcompression rate; and an image recording unit configured to record thecompressed combined image data, left-eye image data, and right-eye imagedata in the same image file.

According to the imaging apparatus of the present invention, it ispossible to achieve efficient file management by recording a left-eyeimage, a right-eye image, and a combined image in one file, and minimizethe file size of the recorded images.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates an exemplary configuration of animaging element applied by the imaging apparatus.

FIG. 2A and FIG. 2B illustrate exemplary configurations of a pixel of animaging element.

FIG. 3 is a conceptual diagram which depicts how light fluxes emittedfrom the exit pupil of a photographic lens enter an imaging element.

FIG. 4 illustrates an exemplary configuration of the imaging apparatusof the present embodiment.

FIG. 5 illustrates an example of a file structure of image data.

FIG. 6 is a flowchart which serves to describe an example of an imagefile generation process.

FIG. 7 illustrates an example of a JPEG image compression rate settingscreen.

FIG. 8 is a flowchart which serves to describe an example of processingfor JPEG compression rate determination.

FIG. 9 is a flowchart which serves to describe an example of processingfor JPEG compression rate determination.

FIG. 10 illustrates an example of a file structure of image data.

FIG. 11 illustrates an example of a file structure of image data.

DESCRIPTION OF THE EMBODIMENTS

As stated above, to view a stereoscopic image, it is necessary toutilize a left-eye image and a right-eye image, and recording theseimages in the same file enables efficient file management.

Now, an imaging apparatus provided with an imaging element havingmultiple photodiodes corresponding to a single micro lens mayconceivably conduct the following processing. For example, this imagingapparatus utilizes a combined image that is generated by combining aleft-eye image and a right-eye image for a two-dimensional displayimage, and utilizes the left-eye image and right-eye image for athree-dimensional display image. Recording these images in the same fileenables efficient file management, but there is the problem that threeimages are recorded in the same file, increasing the file size.According to the imaging apparatus of the present embodiment describedbelow, the aforementioned problem can be solved.

FIG. 1 schematically illustrates an exemplary configuration of animaging element applied by the imaging apparatus of the presentembodiment. An imaging element 100 includes a pixel array 101, avertical selection circuit 102 which selects a row in the pixel array101, and a horizontal selection circuit 104 which selects a column inthe pixel array 101. A read-out circuit 103 reads out the signal of apixel selected by the vertical selection circuit 102 from among thepixels in the pixel array 101. The read-out circuit 103 has a memory forstoring the signals, a gain amplifier, an AD converter, and the like foreach column.

A serial interface (SI) unit 105 determines the operational mode of eachcircuit according to instructions from an external circuit. The verticalselection circuit 102 sequentially selects multiple rows of the pixelarray 101, and extracts the pixel signals to the read-out circuit 103.The horizontal selection circuit 104 sequentially selects the multiplepixel signals read by the read-out circuit 103 for each column. Inaddition to the components shown in FIG. 1, the imaging element 100 has,for example, a timing generator that provides a timing signal to thevertical selection circuit 102, the horizontal selection circuit 104,the read-out circuit 103, and the like, and a control circuit, but adetailed description thereof will be omitted.

FIGS. 2A and 2B are diagrams illustrating an exemplary configuration ofa pixel of the imaging element 100. FIG. 2A schematically shows theconfiguration of a single pixel. FIG. 2B shows the arrangement of thepixel array 101. A pixel 201 shown in FIG. 2A has a micro lens 202serving as an optical element, and multiple photodiodes (hereinafterabbreviated as “PD”) serving as light-receiving elements.

FIG. 2A shows an example where a left-side PD 203 and a right-side PD204 are provided in one pixel, but three or more (e.g., four or nine)PDs may also be used. The PD 203 photoelectrically converts a receivedlight flux to output a left-eye image. The PD 204 photoelectricallyconverts a received light flux to output a right-eye image. In thedescription which follows, the PD of the left side is also denoted asthe left-side PD, and the PD of the right side is also denoted as theright-side PD. In addition to the illustrated components, the pixel 201includes, for example, a pixel amplifier which extracts PD signals tothe read-out circuit 103, a row selection switch, and a PD signal resetswitch.

In order to provide a two-dimensional image, the pixel array 101 isconfigured in a two-dimensional array layout as with the multiple pixels301 to 304 shown in FIG. 2B. The PDs 301L, 302L, 303L, and 304Lrespectively correspond to the PD 203 of FIG. 2A, and the PDs 301R,302R, 303R, and 304R respectively correspond to the PD 204 of FIG. 2A.That is, the imaging apparatus of the present embodiment is providedwith an imaging element which includes multiple pixels that respectivelyhave a first photoelectric conversion unit (PD 203) for outputting aleft-eye image, and a second photoelectric conversion unit (PD 204) foroutputting a right-eye image.

Next, a description is given of the light receiving of the imagingelement 100 having the pixel configuration shown in FIG. 2B. FIG. 3 is aconceptual diagram depicting how light fluxes emitted from an exit pupilof a photographic lens enter the imaging element 100.

The pixel array 101 has a micro lens 402, a color filter 403, and PDs404 and 405. The PDs 404 and 405 respectively correspond to the PDs 203and 204 of FIG. 2A.

In FIG. 3, the center of the light flux that is emitted from an exitpupil 406 of a photographic lens to the micro lens 402 constitutes anoptical axis 409. The light emitted from the exit pupil 406 enters theimaging element 100 centered on the optical axis 409. The partialregions 407 and 408 are regions of the exit pupil 406 of thephotographic lens. Light beams 410 and 411 are the outermost peripherallight beams of the light passing through the partial region 407. Lightbeams 412 and 413 are the outermost peripheral light beams of the lightpassing through the partial region 408.

Among the light fluxes emitted from the exit pupil 406, the upper lightflux enters the PD 405 and the lower light flux enters the PD 404, withthe optical axis 409 serving as the boundary. In short, the PD 404 andthe PD 405 each receive a light flux from a different region of the exitpupil of the photographing optical system. As each light-receivingelement detects light of a different region in the exit pupil in thismanner, they obtain photographic images of respectively different shape,in a condition where light from a point light source is captured in ablurred state.

FIG. 4 is a diagram which illustrates an exemplary configuration of theimaging apparatus of the present embodiment. With reference to FIG. 4, adescription will be given of an exemplary application of the imagingelement 100 shown in FIG. 1 to a digital camera that is an imagingapparatus. A lens unit 501 constituting the imaging optical systemfocuses the light from an object on an imaging element 505. The imagingelement 505 corresponds to the imaging element 100 shown in FIG. 1, andhas the pixel configuration shown in FIG. 2B.

A lens drive device 502 performs zoom control, focus control, diaphragmcontrol, and the like. A mechanical shutter 503 is controlled by ashutter drive device 504. The imaging element 505 converts an objectimage focused by the lens unit 501 into an image signal. An imagingsignal processing circuit 506 performs various types of processing orcorrection on the image signals output by the imaging element 505. Atiming generator 507 outputs timing signals required by the imagingelement 505 or the imaging signal processing circuit 506.

A system control unit 509 is a control unit that performs variouscomputations, and that controls the imaging apparatus in its entirety.Processing by the system control unit 509 is realized by which a CPU(Central Processing Unit) (not shown) provided in the imaging apparatusexecutes programs. A memory unit 508 includes a memory that temporarilystores image data. A recording medium control interface unit(hereinafter abbreviated as “I/F unit”) 510 is provided forrecording/reading image data or the like on/from a recording medium 511.The recording medium 511 which is detachable from the imaging apparatusis a semiconductor memory or the like. An external I/F unit 512transmits/receives data to/from an external device.

The imaging signal processing circuit 506 performs various types ofcorrection and image processing with respect to image signals output bythe imaging element 505. The memory unit 508 temporarily stores outputdata from the imaging signal processing circuit 506, combined imagesgenerated by the image combining circuit 513, and the like. The imagecombining circuit 513 generates a combined image. An image compressioncircuit 514 functions as an image compression unit that compresses imagesignals. The display unit 515 displays various types of information aswell as captured images. The control method of the imaging apparatus ofthe present embodiment is realized by the function of the eachprocessing unit provided in the imaging apparatus shown in FIG. 4.

Next, a description is given of the operations of the digital cameraduring photographing. When the main power supply is turned on, the powersupply of a control system circuit unit is turned on, and the powersupply of an imaging processing system circuit such as the imagingsignal processing circuit 506 is also turned on. When a user operates arelease button (not shown), the system control unit 509 computes focusstate detection based on data from the imaging element 505 to calculatethe distance from the imaging apparatus to the object. Subsequently, thelens drive device 502 drives the movable lens of the lens unit 501, andthe system control unit 509 determines whether or not the focus state isin-focus.

When the system control unit 509 determines that the focus state is notin-focus, it controls the driving of the lens unit 501 again to executefocus state detection processing. For computation of the distance to theobject, in addition to a method for calculating the distance from dataof the imaging element 505, a method may also be used which uses adistance measuring dedicated device (not shown). The system control unit509 initiates photographing operation after determining that the focusstate is in-focus. When the photographing operation is completed, theimaging signal processing circuit 506 processes the image signal outputby the imaging element 505, and the system control unit 509 controls thewriting of the image data to the memory unit 508.

Image data output by the imaging element 505 is output as image signalsfrom the multiple PDs. In the example shown in FIG. 2B, image signalsare output in the sequence of PD 301L, 301R, 302L, 302R, 303L, 303R,304L, and 304R. The image signal processing circuit 506 conducts imageprocessing by allocating the imaging data output by the imaging element505 to left-eye image data and right-eye image data. Left-eye image datais image data obtained as a result of selecting and processing only theoutput (left-eye image signals) from the left-side PDs 301L, 302L, 303L,and 304L shown in FIG. 2B.

Right-eye image data is image data obtained as a result of selecting andprocessing only the output (right-eye image signals) from the right-sidePDs 301R, 302R, 303R, and 304R shown in FIG. 2B. Left eye image data andright-eye image data are separately retained in the memory unit 508.

The image combining unit 513 reads the respective data for a left-eyeimage and a right-eye image retained in the memory unit 508, andgenerates a combined image. The generated combined image data is storedin the memory unit 508. Image processing executed by the image combiningunit 513 is processing in which an additive mean value for each pixel ofa left-eye image and a right-eye image is calculated. Accordingly, theshape of the object is interpolated by this image processing. Thecombined image generated thereby has a shape reflecting the shape of anobject. For example, in the case where the shape of the object iscircular, and where the left-eye image and the right-eye image aresemicircular, the combined image will have a circular shape identical tothe shape of the object. That is, the imaging signal processing circuit506 and the image combining unit 513 function as an image generationunit that executes the following processing. This image generation unitgenerates left-eye image data and right-eye image data forthree-dimensional display based on the image signals output by theimaging element 505, and generates combined image data fortwo-dimensional display by additively combining the pertinent generatedleft-eye image data and right-eye image data.

Even when photographing is conducted with the imaging element 505 in astate where the shape of the object is different between a left-eyeimage and a right-eye image, the shape of the object image isinterpolated by the image processing through the image combining unit513, resulting in generation of image data with the correct shape. Notethat the image signal processing circuit 506 may also combine theleft-eye image and the right-eye image after image processing has beenconducted.

By means of the above-described processing, the memory unit 508 stores aleft-eye image, a right-eye image, and a combined image generated by theimage combining circuit 513. Next, the system control unit 509 reads therespective images stored in the memory unit 508. The image compressioncircuit 514 then conducts JPEG compression of the images that are read,and again writes them into the memory unit 508 as display images.Thereafter, the system control unit 509 records the images written intothe memory unit 508 in a detachable recording medium 511 such as asemiconductor memory via the recording medium control I/F unit 510. Notethat the system control unit 509 may also conduct direct input into anexternal computer or the like via the external I/F unit 512.

First Embodiment

A first embodiment is described below. An imaging apparatus of the firstembodiment compresses a combined image at a predetermined firstcompression rate, and compresses a left-eye image and a right-eye imageat a second compression rate that is higher than the first compressionrate

FIG. 5 is a diagram which illustrates an exemplary file structure ofimage data recorded by the imaging apparatus of the first embodiment.The imaging apparatus of the first embodiment records a left-eye image,a right-eye image, and a combined image in the memory unit 508 as animage data file 600. The image data file 600 has a DCF image data fileformat that is generically used in digital cameras. Specifically, theimage data file 600 has the respective regions of a DCF header section610, a thumbnail image section 620, and a JPEG image section 630.

The DCF header section 610 is a region in which DCF header informationis stored, and is given a predetermined data size. The DCF headerinformation contains metadata A (611) such as photographing informationand parameters related to image data stored in the JPEG image section630. The DCF header information also contains an offset value B (612)corresponding to a combined thumbnail image, an offset value C (613)corresponding to a left-eye thumbnail image, and an offset value D (614)corresponding to a right-eye thumbnail image. The DCF header informationfurther contains an offset value E (615) corresponding to a combinedJPEG image, an offset value F (616) corresponding to a left-eye JPEGimage, and an offset value G (617) corresponding to a right-eye JPEGimage. The starting positions of the respective image data are specifiedby the offset values B to G.

The thumbnail image section 620 stores thumbnail images which are usedwhen the imaging apparatus conducts multiple image (index) display.Thumbnail images are images which have been resized by thinning JPEGimages and the like stored in the JPEG image section 630. The thumbnailimage section 620 has a combined thumbnail image section 621, a left-eyethumbnail image section 622, and a right-eye thumbnail image section623.

The combined thumbnail image section 621 records a thumbnail image whichis used when the imaging apparatus conducts two-dimensional display. Theleft-eye thumbnail image section 622 and the right-eye thumbnail imagesection 623 respectively record a left-eye thumbnail image and aright-eye thumbnail image which are used when the imaging apparatusconducts three-dimensional display.

The JPEG image section 630 is a region for storing JPEG images obtainedby compression after the imaging apparatus has conducted imageprocessing of raw images. JPEG images can be applied in numerous commonapplications.

The JPEG image section 630 has a combined JPEG image section 631, aleft-eye JPEG image section 632, and a right-eye JPEG image section 633.The combined JPEG image section 631 records a JPEG image which is usedwhen the imaging apparatus conducts two-dimensional display. Theleft-eye JPEG image section 632 and right-eye JPEG image section 633respectively record a left-eye JPEG image and a right-eye JPEG imagewhich are used when the imaging apparatus conducts three-dimensionaldisplay.

With the file structure shown in FIG. 5, while three types of JPEGimages are stored in a single file, and the respective JPEG images areused for different applications. Accordingly, the imaging apparatusdetermines the compression rate of the respective JPEG images accordingto the application for which the JPEG image is used.

Note that the file structure of the image files recorded by the imagingapparatus is not limited to the file structure shown in FIG. 5. Theimaging apparatus may also record image files having a file structurethat modifies the sequence of the respective images shown in FIG. 5,according to various applications such as image editing.

FIG. 6 is a flowchart describing an example of image file generationprocessing by the imaging apparatus of the first embodiment. In thisexample, the imaging apparatus generates a DCF file as the image file.First, the system control unit 509 obtains raw data from the right-sidePDs and the left-side PDs of the imaging element 505. That is, thesystem control unit 509 obtains raw data from all of the PDs with whichthe imaging element is provided, and stores it in the memory unit 508(step S701). In the present embodiment, the raw data obtained from allof the PDs is denoted as “all PD raw data”.

Next, the imaging signal processing circuit 506 develops the all PD rawdata of the memory unit 508 using development parameters for raw images,generates raw image data, and arranges it in the memory unit 508 (stepS702). As this raw image data uses signals read out from all of the PDs,it can be recognized as image data of the correct shape even withobjects for which the light of a point light source is captured in ablurred manner. In this example, for purposes of convenience, the imagegenerated from all of the PDs uniting right-side PDs and left-side PDsis considered as the combined image. The raw image data generated instep S702 is also denoted as “combined raw image data”. That is, insteadof additively combining left-eye image data and right-eye image data,the imaging signal processing circuit 506 generates combined image datafor two-dimensional display based on all of the image signals output bythe imaging element 505.

Next, the imaging signal processing unit 506 develops the combined rawimage data obtained in step S702 using the development parameters forJPEG images. Furthermore, the image compression circuit 514 generates acombined JPEG image and a combined thumbnail image by subjecting thepertinent development results to image compression at a prescribedcompression rate (step S703). The processing to determine thecompression rate to be applied in step S703 is described below.

Next, the system control unit 509 judges whether to generate an imagefile for two-dimensional display or an image file for three-dimensionaldisplay (step S704). The 3D image file defined here is a file with aformat enabling treatment as both a two-dimensional image and athree-dimensional image. In the case where the system control unit 509judges that an image file for three-dimensional display is to begenerated, the process advances to step S705.

In step S705, the system control unit 509 inputs the all PD raw dataobtained in step S701 into the imaging signal processing circuit 506,and the imaging signal processing circuit 506 extracts the raw datacorresponding to the right-side PDs. By developing the extracted rawdata using the development parameters for raw images, the imaging signalprocessing circuit 506 generates right-eye raw image data, and arrangesit in the memory unit 508 (step S705).

Next, the imaging signal processing circuit 506 develops the right-eyeraw image data in the memory unit 508 using the development parametersfor JPEG images. The image compression circuit 514 then subjects thepertinent development results to image compression at a prescribedcompression rate, thereby generating a right-eye JPEG image and aright-eye thumbnail image (step S706). The processing to determine thecompression rate to be applied in step S706 is described below.

Next, the system control unit 509 inputs the all PD raw data obtained instep S701 into the imaging signal processing circuit 506, and theimaging signal processing circuit 506 extracts the raw datacorresponding to the left-side PDs. By developing the extracted raw datausing the development parameters for raw images, the imaging signalprocessing circuit 506 generates left-eye raw image data, and arrangesit in the memory unit 508 (step S707).

Next, the imaging signal processing circuit 506 develops the left-eyeraw image data in the memory unit 508 using the development parametersfor JPEG images. The image compression circuit 514 then subjects thepertinent development results to image compression at a prescribedcompression rate, thereby generating a left-eye JPEG image and aleft-eye thumbnail image (step S708). The processing to determine thecompression rate to be applied in step S708 is described below.

Next, the system control unit 509 generates a DCF header (step S709).That is, the system control unit 509 stores information in, for example,the DCF header section 610 shown in FIG. 5. Specifically, the systemcontrol unit 509 sets information including exposure time and shutterspeed during photographing in the metadata A (611) for photographinginformation, parameters, and the like of FIG. 5. The system control unit509 also calculates the offset values B-G (612-619) corresponding toeach image based on the data size of each image generated until theprocessing of step S709, and sets the pertinent offset value.

Next, the system control unit 509 generates and records a DCF filehaving, for example, the structure shown in FIG. 5 using the DCF headergenerated in step S709, and the respective JPEG images and thumbnailimages (step S710). That is, the system control unit 509 functions as animage recording unit that records the compressed combined image data,left-eye image data, and right-eye image data in the same image file.

In the case where the system control unit 509 judges in theaforementioned step S704 that an image file for two-dimensional displayis to be generated, the process advances to step S709, and the followingDCF header preparation processing is executed. The system control unit509 sets the metadata A (611) for photographing information, parameters,and the like. The system control unit 509 also calculates and sets theoffset values B (612) and E (615) corresponding to the combined images.

The system control unit 509 sets a prescribed value indicating that animage is not included in the image data file 600 for offset values C(613), D (614), F (616), and G (617) corresponding to the left-eyeimages and right-eye images. It is also acceptable to employ a methodwhere the system control unit 509 does not record the offset value itemitself. Subsequently, in step S710, the system control unit 509generates a DCF file composed of a DCF header section, a combined JPEGimage, and a combined thumbnail image (step S710), and processingterminates. As the file structure of an image file for two-dimensionaldisplay is identical to the structure of a conventional DCF file,description thereof is omitted here.

FIG. 7 is a diagram which illustrates an example of a JPEG imagecompression rate setting screen. The JPEG image compression rate settingscreen is a screen on which the compression rate of JPEG images is set.In the case where the user selects the compression rate “high”, thesystem control unit 509 determines that a high compression rate as thecompression rate set by the user. The image compression circuit 514compresses the JPEG images at the high compression rate, whereby JPEGimages of small file size are obtained.

In the case where the user selects compression the rate “low,” thesystem control unit 509 determines that low compression rate as thecompression rate selected by the user. The image compression circuit 514compresses JPEG images at the low compression rate, whereby JPEG imagesof large file size are obtained.

FIG. 8 is a flowchart which serves to describe the processing thatdetermines the JPEG compression rate in the first embodiment. In thefollowing description, a user conducts selection operations on the JPEGimage compression rate setting screen so that a low compression rate isset as the compression rate to be used in compressing JPEG images.First, a system control unit 509 judges the type of JPEG image that isthe object of compression processing (step S1001).

In the case where the JPEG image that is the object of compressionprocessing is a “combined image” stored in the combined JPEG imagesection 631, the system control unit 509 determines that the compressionrate set by the user—i.e., a low compression rate—is the compressionrate to be used in compressing the pertinent JPEG image. The compressionrate set by the user is the first compression rate.

Compared to a right-eye image or left-eye image, a combined image is animage that has no sense of discomfort for the viewer. Accordingly, acombined image has diverse applications, and may be used when conductingtwo-dimensional display in common applications, or when conductingretouch processing, or when conducting printing output, or the like.Thus, in step S1002, the image compression circuit 514 compresses thecombined image at the compression rate set by the user (step S1002). Theprocessing in step S1002 corresponds to the image compression processingin step S703 of FIG. 6 described above.

In the case where the JPEG image that is the object of compressionprocessing is a “left-eye image” stored in the left-eye JPEG imagesection 632 or a “right-eye image” stored in the right-eye JPEG imagesection 633, the system control unit 509 performs the followingprocessing. The system control unit 509 determines that a compressionrate that is higher than the compression rate set by the user is thecompression rate to be used to compress the pertinent JPEG image. Thedetermined compression rate is the second compression rate. The systemcontrol unit 509 adopts, for example, a compression rate that is largerby a predetermined value than the compression rate set by the user.

The left-eye image and the right-eye image are used when the imagingapparatus displays a three- dimensional image on the display monitor.The resolution of a common display monitor is lower than the resolutionof the stored image data. Moreover, as the image data is displayed as athree-dimensional image, there is little obtainment of the effect ofrecording at a low compression rate even if the image data is stored ata low compression rate. Consequently, the image compression circuit 514conducts compression at a second compression rate that is higher thanthe compression rate set by the user (step S1003). The processing instep S1003 corresponds to the image compression processing in steps S706and S708 of FIG. 6.

The imaging apparatus of the first embodiment performs compressionprocessing of a “combined image” that has diverse applications accordingto a compression rate set by the user, and performs compressionprocessing of a left-eye image and a right-eye image that have limitedapplications at a compression rate that is higher than the compressionrate set by the user. Therefore, according to the imaging apparatus ofthe first embodiment, it is possible to achieve efficient filemanagement by recording a combined image, a left-eye image, and aright-eye image in the same file, to minimize the file size of recordedimages, and to prevent unnecessary increases in file size.

Second Embodiment

Next, the imaging apparatus of a second embodiment is described. Afeature of JPEG compression is that even when the same compressionparameters are employed, the data size after compression will vary dueto the image data content. For example, the data size after compressionincreases in the case where numerous high-frequency components arecontained in the image data, and conversely, the data size aftercompression decreases in the case where there are few high-frequencycomponents. Accordingly, the imaging apparatus of the second embodimentdetermines a compression rate for a left-eye image and a right-eye imageaccording to the compression result of a combined image for whichcompression processing was previously conducted.

FIG. 9 is a flowchart which serves to explain an example of processingto determine a JPEG compression rate in the second embodiment. First,the system control unit 509 judges the type of JPEG image that is theobject of compression processing (step S1101).

In the case where the JPEG image that is the object of compressionprocessing is a “combined image” stored in the combined JPEG imagesection 631, the system control unit 509 determines that a compressionrate set by the user—i.e., a predetermined first compression rate—is thecompression rate to be used in compressing the pertinent JPEG image. Theimage compression circuit 514 then compresses the combined image at thecompression rate set by the user (step S1102).

Next, the system control unit 509 determines in step S1103 a compressionrate to be used when compressing a left-eye image and a right-eye imagebased on the data size after compression of the combined image (stepS1103). In this example, the imaging apparatus has stored in advance atable—in which is set correspondence information between a data sizeafter compression and a second compression rate corresponding thereto—ina memory unit. The system control unit 509 references the pertinenttable to determine a compression rate corresponding to a data size aftercompression as the first compression rate. Specifically, the systemcontrol unit 509 obtains the data size of combined image data aftercompression as the combined image data compression result. The systemcontrol unit 509 then determines a second compression rate forcompression of left-eye image data and right-eye image data based on thedata size of the obtained combined image data after compression and thecorrespondence information in the aforementioned table. Note that thesystem control unit 509 may also determine a compression rate to be usedin compressing a left-eye image and a right-eye image so that, forexample, the total data volume of a combined image, a left-eye image,and a right-eye image after compression is equal to or less than aprescribed value.

In the case where the JPEG image that is the object of compressionprocessing is a “left-eye image” stored in the left-eye JPEG imagesection 632 or a “right-eye image” stored in the right-eye JPEG imagesection 633, the process advances to step S1103. The system control unit509 then compresses the left-eye image or right-eye image at thecompression rate determined in step S1103 (step S1104).

The imaging apparatus of the second embodiment performs compressionprocessing of a combined image that has diverse applications at acompression rate set by the user, and performs compression processing ofa left-eye image and a right-eye image that have limited applications ata compression rate that is adaptively determined according to thecompression result of the combined image. Therefore, according to theimaging apparatus of the second embodiment, it is possible to achieveefficient file management by recording a combined image, a left-eyeimage, and a right-eye image in the same file, and to prevent increasesin file size regardless of the data content of a previously compressedcombined image.

Other Embodiments

With the imaging apparatus of the first and second embodiments, when theall PD raw data combining the right-side PDs and the left-side PDs isinput into the imaging signal processing circuit 506, it becomespossible to output combined image data, right-eye image data, andleft-eye image data, respectively. With the imaging apparatus of anotherembodiment, the raw data of left-side PDs and the raw data of theright-side PDs are respectively inputted into the imaging signalprocessing circuit 506, right-eye image data and left-eye image data areoutput, and combined image data is output by passing the right-eye imagedata and the left-eye image data through the image combining circuit513.

By means also of an imaging apparatus having this configuration, it ispossible to store three types of JPEG images with the file structureshown in FIG. 5 in one file, and determine the respective compressionrates of the JPEG images by the processing described in FIG. 8 and FIG.9.

With the imaging apparatus of other embodiments, it is also possible togenerate a combined image, a right-eye image, and a left-eye image asraw image file structures. FIG. 10 and FIG. 11 are diagrams whichillustrate exemplary image data file structures of other embodiments. Animage data file 900 shown in FIG. 10 has the respective regions of a rawheader section 910, a thumbnail image section 920, a display-use JPEGimage section 930, and a raw image section 940.

The raw header section 910 is a region in which raw header informationis stored, and is given a predetermined data size. The raw headerinformation contains metadata A (911) such as imaging information andparameters related to the image data stored in the raw image section940.

The raw header information includes an offset value B (912)corresponding to a combined thumbnail image, an offset value C (913)corresponding to a left-eye thumbnail image, and an offset value D (914)corresponding to a right-eye thumbnail image. The raw header informationalso includes an offset value E (915) corresponding to a JPEG image forcombined display, an offset value F (916) corresponding to a JPEG imagefor left-eye display, and an offset value G (917) corresponding to aJPEG image for right-eye display. The raw header information furtherincludes an offset value H (918) corresponding to a right-eye raw image,and an offset value I (919) corresponding to a left-eye raw image. Thestarting positions of the respective image data are specified by theseoffset values B to I.

The configuration of the thumbnail image section 920 is identical tothat of the thumbnail image section 620 shown in FIG. 5. The thumbnailimage section 920 has a combined thumbnail image section 921, a left-eyethumbnail image section 922, and a right-eye thumbnail image section923.

The display-use JPEG image section 930 is a region in which a JPEG imageis stored, where the JPEB image is obtained by processing the raw imagestored in the raw image section 940 by a prescribed compression method.The display-use JPEG image section 930 has a combined display-use JPEGimage section 931, a left-eye display-use JPEG image section 932, and aright-eye display-use JPEG image section 933. The combined display-useJPEG image section 931 records a JPEG image used when performingtwo-dimensional display. The left-eye display-use JPEG image section 932and the right-eye JPEG image section 633 respectively record a left-eyeimage for JPEG display and a right-eye image for JPEG display used whenperforming three-dimensional display.

The raw image section 940 is a region in which raw data is stored, wherethe raw data is output so that the output from the imaging element 505does not degrade. Raw data is data which serves to achieve moresophisticated image editing that conforms to high-quality printing oruser purposes by performing reproduction (development) processing ofimages by means of external devices such as personal computers.

Raw images are not suited to the display of images of large data volumeor the like. Accordingly, the display-use JPEG section 930 is used inthe case where the imaging apparatus actually performs three-dimensionalstereoscopic display. Consequently, the raw image section 940 recordsthe raw data serving as raw material in order to minimize data volume.Specifically, left-eye raw data obtained from left-side PDs is recordedin the left-eye raw image section 941, and right-eye raw data obtainedfrom right-side PDs is recorded in the right-eye raw image section 942.

The imaging apparatus may also store the three types of JPEG images infiles having the file structure shown in FIG. 11. In the example shownin FIG. 11, the raw image section 940 has a raw image section 943. Theimaging apparatus records raw data that is not divided for left eye useand right eye use in the raw image section 943.

Preferred embodiments of the present invention have been describedabove, but the present invention is not limited to these embodiments,and is capable of a variety of modifications and alterations within thescope of its intent.

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiments, and by a method, the steps of whichare performed by a computer of a system or apparatus by, for example,reading out and executing a program recorded on a memory device toperform the functions of the above-described embodiments. For thispurpose, the program is provided to the computer, for example, via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2012-001336 filed Jan. 6, 2012, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An imaging apparatus comprising: an imagingelement provided with pixels which respectively have multiplephotoelectric conversion units that generate image signals byphotoelectrically converting light fluxes that transit different regionsof exit pupil of an imaging optical system to one micro lens; an imagegeneration unit configured to generate left-eye image data and right-eyeimage data for three-dimensional display based on image signals outputby the imaging element, and generate combined image data fortwo-dimensional display by additively combining the generated left-eyeimage data and right-eye image data; an image compression unitconfigured to compress the combined image data at a predetermined firstcompression rate, and compress the left-eye image data and the right-eyeimage data at a second compression rate that is higher than the firstcompression rate; and an image recording unit configured to record thecompressed combined image data, left-eye image data, and right-eye imagedata in the same image file.
 2. The imaging apparatus according to claim1, wherein the pixels respectively have a first photoelectric conversionunit for outputting left-eye image signals, and a second photoelectricconversion unit for outputting right-eye image signals, and wherein theimage generation unit generates the left-eye image data based on theleft-eye image signals, and generates the right-eye image data based onthe right-eye image signals.
 3. The imaging apparatus according to claim1, wherein the image generation unit generates the combined image datafor two-dimensional display based on all image signals output by theimaging element, instead of by additively combining the generatedleft-eye image data and right-eye image data.
 4. The imaging apparatusaccording to claim 1, wherein the image compression unit determines thesecond compression rate when compressing the left-eye image data and theright-eye image data based on the compression result of the combinedimage data.
 5. The imaging apparatus according to claim 1, wherein theimage compression unit obtains the data size of combined image dataafter compression as the compression result of the combined image data,and determines the second compression rate when compressing the left-eyeimage data and the right-eye image data based on the data size of theobtained combined image data after compression, and correspondenceinformation between data sizes after compression of the combined imagedata and second compression rates that is stored in advance in a memoryunit.
 6. A control method of an imaging apparatus comprising pixelswhich respectively have multiple photoelectric conversion units thatgenerate image signals by photoelectrically converting light fluxes thattransit different regions of an exit pupil of an imaging optical systemto a single micro lens, the control method comprising: generatingleft-eye image data and right-eye image data for three-dimensionaldisplay based on image signals output by the imaging element, andgenerating combined image data for two-dimensional display by additivelycombining the generated left-eye image data and right-eye image data;compressing the combined image data at a predetermined first compressionrate, and compressing the left-eye image data and the right-eye imagedata at a second compression rate that is higher than the firstcompression rate; and storing the compressed combined image data,left-eye image data, and right-eye image data in the same image file.